Loop Series and Bethe Variational Bounds in Attractive Graphical Models
نویسندگان
چکیده
Variational methods are frequently used to approximate or bound the partition or likelihood function of a Markov random field. Methods based on mean field theory are guaranteed to provide lower bounds, whereas certain types of convex relaxations provide upper bounds. In general, loopy belief propagation (BP) provides often accurate approximations, but not bounds. We prove that for a class of attractive binary models, the so–called Bethe approximation associated with any fixed point of loopy BP always lower bounds the true likelihood. Empirically, this bound is much tighter than the naive mean field bound, and requires no further work than running BP. We establish these lower bounds using a loop series expansion due to Chertkov and Chernyak, which we show can be derived as a consequence of the tree reparameterization characterization of BP fixed points.
منابع مشابه
New Understanding of the Bethe Approximation and the Replica Method
In this thesis, new generalizations of the Bethe approximation and new understanding of the replica method are proposed. The Bethe approximation is an efficient approximation for graphical models, which gives an asymptotically accurate estimate of the partition function for many graphical models. The Bethe approximation explains the well-known message passing algorithm, belief propagation, whic...
متن کاملWhat Cannot be Learned with Bethe Approximations
We address the problem of learning the parameters in graphical models when inference is intractable. A common strategy in this case is to replace the partition function with its Bethe approximation. We show that there exists a regime of empirical marginals where such Bethe learning will fail. By failure we mean that the empirical marginals cannot be recovered from the approximated maximum likel...
متن کاملThe Bethe Partition Function of Log-supermodular Graphical Models
Sudderth, Wainwright, and Willsky conjectured that the Bethe approximation corresponding to any fixed point of the belief propagation algorithm over an attractive, pairwise binary graphical model provides a lower bound on the true partition function. In this work, we resolve this conjecture in the affirmative by demonstrating that, for any graphical model with binary variables whose potential f...
متن کاملPartition function loop series for a general graphical model: free energy corrections and message-passing equations
A loop series expansion for the partition function of a general statistical model on a graph is carried out. If the auxiliary probability distributions of the expansion are chosen to be a fixed point of the belief-propagation equation, the first term of the loop series gives the Bethe-Peierls free energy functional at the replicasymmetric level of the mean-field spin glass theory, and correctio...
متن کاملBeyond Log-Supermodularity: Lower Bounds and the Bethe Partition Function
A recent result has demonstrated that the Bethe partition function always lower bounds the true partition function of binary, logsupermodular graphical models. We demonstrate that these results can be extended to other interesting classes of graphical models that are not necessarily binary or log-supermodular: the ferromagnetic Potts model with a uniform external field and its generalizations a...
متن کامل